notebook.community

Edit and run

Download and Check

If you are running Julia and the IJulia notebook on your own computer, visit

https://github.com/kersulis/IJulia-WPS

and click on "Download ZIP" at the bottom of the toolbar on the right of the page. Alternatively, you can enter this URL into your address bar:

https://github.com/kersulis/IJulia-WPS/archive/master.zip

Now that you have the notebooks, let's make sure your tools are working. Click on the following cell, then choose Cell -> Run or press Ctrl+Enter:





In [ ]:



    


sin(π/2)

If you see 1.0 and no errors, then you just ran Julia code. Great! (This also proves you are not using a Python kernel.)

If anything else happened, please raise your hand! We want to make sure you can interact with Julia throughout the remainder of this session. Even if you didn't work through the installation guide, you can still connect to JuliaBox using the following instructions.

Backup Option: Connecting to JuliaBox

  1. Visit juliabox.org.
  2. Select the Sync tab at the top of the page.
  3. Enter https://github.com/kersulis/IJulia-WPS into the Git Clone URL field.
  4. Leaving the Branch and JuliaBox Folder fields with their default values, click + to load the repository into a folder named IJulia.
  5. Now click on the IJulia tab at the top of the page. If you don't see a new folder called IJulia, refresh your browser (don't worry, no files will be lost).
  6. Enter the IJulia folder and click on one of the notebooks to open it in a new tab. The notebook is editable, and any Julia code cell should execute and show appropriate output.

Any notebook changes you make will be saved on your own computer -- they will have no effect on the repository.

This talk

What I want to accomplish in this talk:

  • Convince you that Julia is an approachable, elegant, and highly capable technical computing language
  • Show, by example, the usefulness of the IJulia notebook.
  • Familiarize you with Julia and the IJulia notebook.
  • Show how easy it is to start using these tools regardless of background (MATLAB, Python, C++).

Six months ago I was running MATLAB on Windows with a student license. The more I read about research workflows, the more convinced I became that there were better ways to go. This past summer I decided to explore many of the tools I was reading about, and it transformed the way I do my research work. Today I run Python and Julia on Ubuntu Linux. Here's why I chose Julia.

Three reasons I chose Julia

  1. The JuMP Julia package effectively replaces the thirty-year-old AMPL modeling language.

  2. Julia is fast.

  3. Julia syntax is natural for MATLAB users. It supports [a b; c d] matrix concatenation. It has many MATLAB functions like sort, setdiff, unique, sparse, and even spy. When you tell it to invert a matrix (using the same A\B syntax MATLAB uses), it figures out the best algorithm to use. You can begin writing Julia code using MATLAB style and gradually mix in Julia tricks as you go along.

Nice side benefits: no more watermarks, nothing proprietary.

My experience so far

Although learning any new language is rocky at first, I have already found myself doing things I could not have done with my former tools. One example is transmission network visualization: I can use Julia and Graphviz to programmatically generate a rich SVG representation of a power network.

The IPython notebook made my transition easy. When I first contemplated switching from MATLAB, I was intimidated by the command line. I couldn't find a comfy, MATLAB-esque development environment for Julia. How could I give up side-by-side script and output or the variable workspace? IPython/IJulia notebooks are the answer. A notebook consists of cells. Each cell can be a header, a markdown cell (which can include $\LaTeX$ equations, bullet lists, and tables), or a code cell. When you write and execute a code cell, its output (whether it be a matrix, some text, or a chart) is displayed inline, so the notebook is self-contained and flows nicely. When you are finished creating a notebook, you can export it in a variety of formats including HTML and $\LaTeX$.

There are a bunch of example notebooks that demonstrate what you can do; they are a great way to learn. The notebooks we will go through here are more domain-specific, and I hope they are helpful for power system researchers. In this session we will consider the following research workflow:

  1. Formulate an optimization problem in $\LaTeX$.
  2. Import data from a MATLAB .mat file (using Julia's MAT package to generate a set of Julia variables).
  3. Use JuMP to solve the optimization problem.
  4. Perform a series of checks to make sure the output makes sense.
  5. Plot the results.

Click Help -> User Interface Tour

Think of the notebook as a document that can interact with your computer. The document relies only on a modern browser for rendering -- it is useful by itself. When you connect the document to a Julia kernel and terminal instance on a computer, however, the document can send any command to the computer and show any output (text or graphics). The document and computer need not be in the same place -- only a channel for message passing is required.

image source

  • Each notebook is composed of cells
  • Two modes:
    • Command Mode for creating or deleting cells, saving or renaming the notebook, and other application-level functions
    • Edit Mode for manipulating text in individual cells
  • Create a cell by:
    • Clicking Insert -> Insert Cell
    • Pressing a or b in Command Mode
    • Pressing Alt+Enter in Edit Mode
  • Delete a cell by:
    • Clicking Edit -> Delete Cell
    • Pressing dd
  • Execute a cell by:
    • Clicking Cell -> Run
    • Pressing Ctrl+Enter

Other functions:

  • Undo last text edit with Ctrl+z in Edit Mode
  • Undo last cell manipulation with z in Command Mode
  • Save notebook with Ctrl+s in Edit Mode
  • Save notebook with s in Command Mode

Though notebooks rely on your browser to work, they do not require an internet connection. The only online tool that is consistently used is MathJax (for math rendering), and it is easy to install MathJax locally.

A note about naming

Try not to let the nomenclature confuse you! Some people say "IPython notebook", and a few (mostly core developers) prefer to say "Jupyter notebook". We will use "IJulia notebook" to refer to a notebook connected to a Julia kernel.

Get comfortable with the notebook

Notebooks are hardly fragile. If you try to close a notebook with unsaved changes, the browser will warn you.

Try the following exercises:

[Exercise]: Close/open

  1. Save the notebook
  2. Copy the address
  3. Close the tab
  4. Paste the address into a new tab (or re-open the last closed tab with Ctrl+Shift+T on Chrome)

The document is still there, and the Julia kernel is still alive! Nothing is lost.

[Exercise]: Zoom

Try changing the magnification of the web page (Ctrl+, Ctrl- on Chrome).

Text and math scale well (so do graphics if you use an SVG or PDF backend).

[Exercise]: MathJax

  1. Create a new cell.
  2. Type an opening \$, your favorite mathematical expression, and a closing \$.
  3. Run the cell to render the $\LaTeX$ expression.
  4. Right-click the rendered expression.

Now that we've discussed the notebook, let's talk about Julia, the language that makes this an IJulia notebook. We'll be learning by doing during this session, but I've also put together a list of helpful resources for future exploration at the end of this notebook.





In [ ]:



    


help(print) # How to get help for a Julia object



    











In [ ]:



    


?print # Alternative syntax

Plotting

There are several Julia plotting packages.

  • PyPlot.jl is a Julia interface to Matplotlib, and should feel familiar to both MATLAB and Python users.
  • Winston and Gadfly are written entirely in Julia. Winston is for general-purpose 2D plotting, and Gadfly concentrates on statistical graphics.
  • Plotly supports Julia.




In [ ]:



    


Pkg.add("PyPlot")



    











In [ ]:



    


using PyPlot

# Example from PyPlot documentation:
x = linspace(0,2*pi,1000)
y = sin(3*x + 4*cos(2*x))
plot(x, y,  color="red", 
            linewidth=2.0, 
            linestyle="--")
title("A sinusoidally modulated sinusoid")

Coming from MATLAB, you keep:

  • Simple matrix syntax
  • Semicolon for supressing output in notebook or REPL




In [ ]:



    


B = [1 2; 3 4];
C = [5 6];
[B; C]
  • Many commonly used functions in Base (default namespace, always available)




In [ ]:



    


A = rand(3, 2); # create random matrix
display(A)

U, S, V = svd(A); # Find singular value decomposition



    











In [ ]:



    


U*diagm(S)*V



    











In [ ]:



    


?sin



    











In [ ]:



    


?sind



    











In [ ]:



    


sin(π/2)



    











In [ ]:



    


?randn



    











In [ ]:



    


t = 1:10;
t[t.<5]
  • Ability to read and write .mat files




In [ ]:



    


Pkg.add("MAT")
using MAT



    











In [ ]:



    


matwrite("test.mat", {
    "a1" => sin(π/2),
    "a2" => [1 2 3]
})



    











In [ ]:



    


vars = matread("test.mat")
println(vars["a1"])
println(vars["a2"])

Just remember:

  • Index arrays with square brackets instead of parentheses.
  • Be aware of types.




In [ ]:



    


typeof(1)



    











In [ ]:



    


typeof(1.0)

Coming from Python, you keep:





In [ ]:



    


Pkg.add("Calculus")
using Calculus
derivative(sin, π/3)



    







A = [ F(x,y,...) for x=rx, y=ry, ... ]





In [ ]:



    


A = [(x, x^2, x + y*im) for x=1:3, y in [1 2]];
print(A)

You can import Python modules using PyCall.jl.





In [ ]:



    


Pkg.add("PyCall")



    











In [ ]:



    


using PyCall
@pyimport numpy as np



    











In [ ]:



    


A = [0.5 -0.5;
    -0.5 0.5] # Create array with Julia

B = np.sign(A) # Pass array to NumPy function

print(B) # Show result



    











In [ ]:



    


# No type confusion!
print("Type of A:\t\tType of B: \n", typeof(A), '\t', typeof(B))

Of course, Julia has its own sign() -- this was just an example.





In [ ]:



    


print(sign(A))

Another example: plotting a random walk





In [ ]:



    


@pyimport numpy.random as npr # Must import submodule
C = npr.normal(0, 1, 1000) # Create array using NumPy function

plot(cumsum(C)) # Using PyPlot.jl from earlier

Types, methods, and multiple dispatch

"Julia programs are organized around multiple dispatch"

Though you don't need to know much about Julia's multiple dispatch to use the language, there is a great deal of power and flexibility here. The :: operator ("is an instance of") may be used to assert object types:





In [ ]:



    


(1+2)::FloatingPoint
#(1+2)::Int

In addition to concrete (and familiar) Float and Int types, Julia has abstract types:





In [ ]:



    


(1 + 1im)::Real
#1::Complex



    











In [ ]:



    


(1 + 1im)::Complex
1::Real

The most important implication of Julia's type system is multiple dispatch: functions take all arguments into consideration when determining which method to use. From Wikipedia:





In [ ]:



    


url = "https://en.wikipedia.org/wiki/Dynamic_dispatch#Single_and_multiple_dispatch"
s = string("<iframe height='450' id='multDis' seamless='seamless' src='",
url,"' width='550' scrolling='no'></iframe>")
display("text/html", s)

Let's look at Julia's Complex() function:





In [ ]:



    


?Complex

Use methods() to see every combination of arguments a particular function can handle (in the notebook, you can also press Tab after a function's opening parenthesis):





In [ ]:



    


methods(methods) # meta alert!



    











In [ ]:



    


methods(Complex)

Let's follow one of these links to see how Julia operates on complex numbers.

A couple exercises

[Exercise]: Extending a function

  1. Try applying abs() to a string. What happens?
  2. Extend abs() so it returns the uppercase version of a string argument.

Hint: you must import a Base function before you can extend it (this prevents accidental mangling of basic functions). Use import Base.abs to do this.





In [ ]:



    


abs("Hello") # try this



    











In [ ]:



    


import Base.abs

# Extend abs() here:
function abs(x::SomeType)
    
end

[Exercise]: Normalize

  1. Write a function called normalize() that normalizes a vector x in place (replacing it with a unit-norm vector having the same direction). The following code cell should get you started.

  2. Now test your function using the given code cell. What happens?

  3. Using Julia's type system, modify your function to "fail gracefully."

Note: Julia has several "modifying functions" that alter their inputs without creating any copies. These all end with an exclamation mark. To scale an array x by some value a in place, use:

scale!(x, a)

More information on functions in Julia

Don't be afraid to kill the kernel and start over.





In [ ]:



    


function normalize(x)
    # Manipulate x here. 
    
end



    











In [ ]:



    


# Use this cell to test your function:
x = [1 2 3]
normalize(x)

Exercise solutions

Extending a function





In [ ]:



    


# Extend Base.abs:
import Base.abs
function abs(x::ASCIIString)
    uppercase(x)
end

Normalize





In [ ]:



    


function normalize(x::Array{Float64}; inplace=false)
    if inplace
        scale!(x, 1/sqrt(sum([i^2 for i in x])))
    else
        x /= sqrt(sum([i^2 for i in x]))
    end
end



    











In [ ]:



    


methods(normalize)



    











In [ ]:



    


x = ones(100000000);



    











In [ ]:



    


@time normalize(x);



    











In [ ]:



    


@time normalize(x, inplace=true);

Julia reference

Try visiting the top few links first.

Resource Description
Julia By Example Nicely categorized series of code snippets
Learn Julia in Y Minutes Broad, concise overview of Julia commands
Matrix Cheatsheet Translation between MATLAB/Octave, Python/NumPy, R, and Julia
Julia community page Links to mailing lists, various projects, and local Julia communities
Julia Docs Official documentation for the Julia Language
JuliaCon More than 20 talks on a broad range of subjects, many with links to slides and other supporting materials
Julia Bloggers Blog aggregator that gathers Julia-related blog posts from several writers

What we've covered so far

We started becoming familiar with the IJulia notebook, a free and powerful tool that marries the display capabilities of modern browsers with the reliability of local code execution. Next, we considered Julia: how to find help, how to plot, and how to approach Julia from MATLAB and Python backgrounds. Finally, we discussed Julia's types and how they are used.

Where to next?

Next we will look at the instanton project, an example of power systems research that will illustrate many of the concepts introduced here.

Content source: kersulis/IJulia-WPS

Similar notebooks:

notebook.community | gallery | about